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ABSTRACT 



This study examined the possible relationship between scores 
on the Nelson-Denny Reading Test (current forms G and H) and performance on 
the Medical College Admissions Test (MCAT) and the United States Medical 
Licensing Examination (USMLE) Step 1 examination scores. Participants were 
730 medical students at a mid-Atlantic university, and for 572 of these 
students, MCAT scores were also available. All four Nelson-Denny Reading Test 
variables (Vocabulary, Comprehension, Total Score, and Reading Rate) were 
positively correlated with the MCAT verbal reasoning score. Nelson-Denny 
Vocabulary and Total Scores were positively correlated with the USMLE Step 1 
score, but the Nelson-Denny Comprehension score was not significantly related 
to the USMLE Step 1 score. MCAT scores have been shown to be correlated with 
USMLE scores. Results suggest that medical schools should consider using an 
index of reading to predict applicants who will achieve success in medical 
school. (Contains 15 references.) (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM033874 



Reading and Medical Students 



1 



as 

VO 



Q 



W 



Running head: READING AND MEDICAL STUDENTS 



Adult Learners: Relationships of Reading, MCAT, 
and USMLE Step 1 Test Results for Medical Students 



Patricia A. Haught 
and 

Richard T. Walls 
West Virginia University 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDl^ATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

^ This document has been reproduced as 
received from the person or organization 
originating it. 



□ Minor changes have been made to 
improve reproduction quality. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



° Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Paper presentation at the annual meeting of the American Educational Research Association, New 
Orleans, April 4, 2002. Correspondence concerning this article should be addressed to Patricia A. 
Haught, Ed. D., PO Box 6122, West Virginia University, Morgantown, WV 26506-6122 or email 
phauRht@wvu.edu . 



ILI 



o 



2 



Reading and Medical Students 



2 



Adult Learners: Relationships of Reading, MCAT, 
and USMLE Step 1 Test Results for Medical Students 

Adult learners continue to refine their reading and other cognitive skills, and reading skills are 
tied to academic achievement as well as test taking. Haught and Walls (in press) reported new reading 
norms for the Nelson-Denny Reading Test for health-care professional students that demonstrate higher 
reading scores for those students than for four-year college graduates. Educators have attempted to 
identify various factors that contribute to academic success for medical students. Horshl and Kozeny 
(1997) reported that high school performance, written entrance examination, admission interview, and 
personality traits may significantly predict academic success for third-year medical students in the Czech 
Republic. Twenty-five studies which examined the relationship of learning style and medical student 
performance were summarized by Hylton and Harman (1996) who concluded that “many of the 
generalizations that have been made about the learning styles of medical students actually have little 
conclusive empirical data to support them” (p. 1411). Harward, Tresolini, Davis, and Kretchmar (1998) 
investigated the value of self-efficacy as a predictor of performance and reported that significant 
associations were observed between different types of educational experience (e.g., reading in a content 
area) and self-efficacy ratings on professional skills. Critical thinking skills as measured by the Watson- 
Glaser Critical Thinking Appraisal were moderately predictive of academic success in pre-clinical 
medical students (Scott & Marker!, 1994). 

Earlier versions of the Nelson-Denny Reading Test (Brown, Fischo, & Hanna, 1993) were found 
to be positively correlated with medical school achievement (Ambrosino, Brading, & Noval, 1974). It is 
generally accepted that medical schools require students with better reading ability because of the 
quantity of material to be read. Flaherty, Rezler, and McGuire (1982) found the Nelson-Deimy Reading 
Test {earlier Form E or Form F) to be a better predictor of clinical reading skills than the reading subtest 
of the Medical College Application Test (MCAT). These researchers encouraged medical school 
admission committees to consider reading ability as part of the admission process. 

Jackson and Brooks (1985) examined possible relationships between the MCAT Reading 
Subtest, the Nelson-Denny Reading Test, and medical school achievement. During orientation to 
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medical school, students completed the Nelson-Denny Reading Test {earlier Forms E or F). The 
criterion variables used to examine the predictive ability of these tests included (a) basic science and 
clinical science GPA and (b) total scores on Part I and Part II of the National Board of Medical 
Examinees (NBME) examination. These authors concluded “The NDRT was a better single predictor 
than the MCAT reading score. . .” (p. 479) with Total Score (one of four Nelson Denny Reading Test 
scores) being the best predictor and Reading Rate being the poorest predictor of academic achievement. 
The Nelson-Denny Reading Test was a better predictor of achievement (GPA and NBME) than the 
MCAT reading subtest. In this study (Jackson & Brooks, 1985) the Nelson-Denny Reading Test Total 
Score was moderately correlated (r == .51) to the MCAT reading score, but a higher correlation {r = .69) 
between these measures was reported earlier by Jackson, Dawson-Saunders, and Jackson (1984). 

There are substantial investments of human and financial resources in medical students. 
Universities and their schools of medicine experience pressures of escalating cost and diminishing 
resources. They seek to minimize their losses by admitting the applicants most likely to succeed in 
medical school. One of the present researchers has observed that some students who appear to know the 
content either (a) score low or (b) fail to pass various standardized tests (e.g., MCAT and USMLE Step 
1 Boards). It is possible that they cannot read competently enough to complete the exam and give an 
accurate representation of their own knowledge. The present investigation examines the possible 
relationship of the Nelson-Denny Reading Test {current Forms G or H) performance with the MCAT 
scores and the USMLE Step 1 Board exam scores. Are there predictive relationships from the Nelson- 
Denny (N-D) Reading Test (using the most recent forms of the test) to (a) MCAT scores and (b) 

USMLE Step 1 Board exam scores? Initially, at the time of application, do scores of N-D Reading 
Vocabulary, N-D Reading Comprehension, N-D Total Reading Score, and N-D Reading Rate 
differentially predict MCAT verbal reasoning, MCAT physical sciences, and MCAT biological sciences 
scores (Research Question 1)7 Two years later, are the Nelson-Denny measures of Reading Vocabulary, 
Comprehension, Total, and Rate predictive of the USMLE basic biomedical sciences score (Research 
Question 2)7 
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Participants 

The participants (A^=730) were students in the School of Medicine at a major Mid- 
Atlantic university. There were 431 males and 299 females. The ethnic composition was 
approximately 90 percent White, 1 percent African-American, 1 percent Hispanic, and 8 percent 
Other (e.g.. Eastern Indian). The testing took place from 1994 through 2001. As an example of 
the composition of medical students in a particular year, (a) 998 people applied to the medical 
school in 1999, (b) 255 applicants were interviewed, and (c) 88 applicants were admitted (51 
males and 37 females). Of those students admitted, the mean undergraduate grade point average 
(GPA) in science courses was 3.60, and the mean overall GPA was 3.68. Average Medical 
College Admission Test (MCAT) scores of admitted applicants were 9.0 in Verbal Reasoning, 
8.9 in Physical Sciences, 9.1 in Biological Sciences, and “P” for Writing. Of those 88 students 
admitted, 89% were from 20 to 23 years of age, and most had an undergraduate degree in the 
biological sciences (48%) or chemistry (22%). Most of the admitted applicants were state 
residents (97%). This medical school does not use reading test performance as a criterion for 
admission. The Nelson-Denny Reading Test is used after admission as a possible “red flag” to 
alert faculty and administrators that a student may experience academic difficulties. This 
investigation was approved by the Institutional Review Board for protection of human subjects 
and was conducted in accordance with American Psychological Association ethical guidelines. 
Instruments 

The Nelson-Denny Reading Test is a standardized instrument for assessing reading 
vocabulary, reading comprehension, and reading rate; the two most current forms are Form G 
and Form H. The N-D Reading Vocabulary score is based on 80 multiple-choice items (15 
minutes). The N-D Reading Comprehension score is based on 38 multiple-choice items (20 
minutes). The N-D Total Reading score is the sum of the vocabulary and comprehension scores. 
The N-D Reading Rate score is the number of words read during the first minute of the 
comprehension part of the test. Some participants (n=572) also completed the Medical College 
Admission Test (MCAT), a standardized, multiple-choice exam designed to assess problem 
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solving, critical thinking, and writing skills, as well as, knowledge of science concepts and 
principles prerequisite to the study of medicine. The MCAT provides separate scores in verbal 
reasoning, physical sciences, biological sciences, and writing (Federation of State Medical Board 
of the United States, 1998). Also, a subset of the participants (n=457) took the United States 
Medical Licensing Examination (USMLE) Part 1 at the end of their second year of study. This 
exam assesses knowledge of key concepts of basic biomedical sciences, with an emphasis on 
principles and mechanisms of health, disease, and modes of therapy (National Board of Medical 
Examiners, 2000). 

Procedure 

The Nelson-Denny Reading Test was administered during orientation to medical school. 
Participants completed either Form G or Form H. Time allotted in the orientation schedule for test 
completion was one hour, and the test was completed in a group setting. They were allowed 1 5 minutes 
for the vocabulary subtest and 20 minutes for the comprehension subtest. The first minute of the 
comprehension part of the Nelson-Denny Reading Test was used to determine reading rate. These 
reading test data are used herein for determining relationships to the MCAT (verbal reasoning, physical 
sciences, and biological sciences) and to the USMLE Step 1 Test score for medical students. 

Results 

To examine whether Nelson-Denny (N-D) Reading Test scores were related to performance on 
other standardized exams, Pearson Product-Moment Correlations were computed on the N-D 
Vocabulary, N-D Comprehension, N-D Total, and N-D Reading Rate raw scores with MCAT scores 
(verbal reasoning, physical sciences, and biological sciences) and the USMLE Step 1 score. The N-D 
Vocabulary score was positively correlated (a) with the MCAT verbal reasoning score (r = .53,/? < .01), 
(b) with the MCAT physical science score (r = 2Q,p < .01), (c) with the MCAT biological science score 
(r = 21, p < .01), and (d) with the USMLE Step 1 score (r = .14, /> < .01). As the N-D Vocabulary score 
increased so did the test scores on the MCAT and the USMLE exam. The strongest relationship was 
between the N-D Vocabulary score and the MCAT verbal reasoning score. The N-D Comprehension 
score also was positively correlated with the MCAT verbal reasoning score {r = A\,p < .01), with the 
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MCAT physical sciences score (r = .13,/? < .01), and with the MCAT biological science score (r = .12,/? 
< .01), but not with the USMLE score (p > .05). Similarly, the strongest observed relationship for N-D 
Comprehension was with the MCAT verbal reasoning score. The N-D Total Reading score was 
positively correlated to the four measures: MCAT verbal reasoning (r = .56, p < .01), MCAT physical 
science score (r = .19,/? < .01), MCAT biological science score (r = .23,/? < .01), and USMLE score (r = 
.13,/? < .01). The N-D Reading Rate score was positively correlated only with the MCAT verbal 
reasoning score (r = .24,/? < .01), but not with any remaining MCAT scores or with the USMLE score 
(p>.05). 

To examine whether the MCAT scores were related to performance on the USMLE STEP 1 
exam score, Pearson Product-Moment Correlations were computed on the MCAT verbal reasoning, 
MCAT physical sciences, and MCAT biological sciences scores and the USMLE Step 1 score. The 
USMLE Step 1 score was positively correlated with all parts of the MCAT. The USMLE Step 1 exam 
score was positively correlated with (a) the MCAT verbal reasoning score (r = .18,/? < .01), (b) the 
MCAT physical sciences score (r = .3 1 ,P< .01), and (c) with the MCAT biological sciences score 
(r = .34,/? < .01). 

Multiple regression analyses were computed to determine whether the components of 
The N-D Reading Test (N-D Vocabulary, N-D Comprehension, N-D Reading Rate) were predictors of 
MCAT and USMLE Step 1 criterion variables. The N-D Total was not included as a predictor in these 
analyses because that total is a combination of N-D Vocabulary and N-D Comprehension. First, when 
MCAT verbal reasoning was the criterion variable and the three Nelson-Denny scores were predictor 
variables, the stepwise multiple regression showed the N-D Vocabulary score (/? < .01), the N-D 
Comprehension score (/? < .01), and the N-D Reading Rate score (p < .05) to be significant predictors. 
Second, when MCAT physical science was the criterion variable and the three Nelson-Denny scores 
were predictor variables, the stepwise multiple regression demonstrated that the N-D Vocabulary score 
was a significant predictor (/? < .01). Third, when MCAT biological science was the criterion variable 
and the three Nelson-Denny scores were predictor variables, the stepwise multiple regression revealed 
that the N-D Vocabulary score was a significant predictor (p < .01). Fourth, when the USMLE Step 1 
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score was the criterion variable and the three Nelson-Denny scores were predictor variables, the stepwise 

regression demonstrated that the N-D Vocabulary score was a significant predictor 

(P<.01). 

The strongest relationship established in the correlation analyses was between N-D Total and 
MCAT verbal reasoning (r = .56). To further examine the effects of the N-D Total as related to crucial 
medical school indices, the N-D Total was divided into quartiles so that approximately one-fourth of the 
participants was in each group. A one-way analysis of variance was computed with the N-D Total 
quartiles as the independent variable for each of four dependent variables (MCAT verbal reasoning, 
MCAT physical sciences, MCAT biological sciences, and the USMLE Step 1 score). 

The analysis of variance for MCAT verbal reasoning yielded F(3, 568) = 65.54, p <.01, and the 
Tukey multiple comparisons test indicated that all quartiles differed from one another except Quartiles 2 
and 3. MCAT verbal reasoning raw scores ranged from 4 to 15. The MCAT verbal reasoning means 
progressed from Quartile 1 to Quartile 4 in an orderly fashion (Q1 = 7.58, Q2 = 8.78, Q3 = 9.15, and Q4 
= 9.94). 

The analysis of variance for MCAT physical sciences produced F(3, 568) = 1.96,p <.01. The 
Tukey multiple comparisons test showed that Quartile 1 differed significantly from Quartiles 3 and 4 
(both p <.05), and no other comparisons were significantly different (p >.05). MCAT physical science 
scores ranged from 4 to 14. The MCAT physical science means showed gradual increases from Quartile 
1 through Quartile 4 (Q1 = 8.10, Q2 = 8.59, Q3 = 8.68, and Q4 = 9.01). 

The analysis of variance for MCAT biological sciences resulted in F(3, 568) = 1 1.10,p <.01, and 
the Tukey multiple comparisons test revealed that Quartile 1 differed from Quartiles 3 and 4, and 
Quartile 2 differed from Quartile 4 (p <.05). No differences were observed between Quartiles 1 and 2, 2 
and 3, and 3 and 4 (p >.05). MCAT biological science means ranged from 3 to 14, and mean scores for 
Quartiles 1, 2, 3, and 4 were 8.60, 8.71, 9.18, and 9.48, respectively. The analysis of variance for 
USMLE Step 1 score was not significant (p >.05). 



Discussion 



Reading and Medical Students 



8 



All four Nelson-Denny Reading Test variables (Vocabulary, Comprehension, Total Score, and 
Reading Rate) were positively correlated with the MCAT verbal reasoning score. The present findings, 
using the current N-D Reading Test Forms G and H, show that the MCAT verbal reasoning score is 
strongly correlated with N-D Vocabulary (r = .53), N-D Comprehension (r = .41), and N-D Total Score 
(r = .56). A smaller, but significant positive correlation was observed between MCAT verbal reasoning 
and the N-D Reading Rate (r = .24). With earlier forms (E and F) of the N-D Reading Test, Jackson and 
Brookes (1985) reported comparable results between the MCAT reading measure and N-D Vocabulary 
(r = .44), N-D Comprehension (r = .48), N-D Total (r = .51), and N-D Reading Rate (r = .09). The 
newer forms (G and H) of the N-D Reading Test yielded somewhat better prediction of MCAT verbal 
reasoning than reported by Jackson and Brooks. Given that a different university was involved 1 5 years 
later, this is indicative that the N-D Reading Test has remained an effective screening or predictive 
instrument. In addition to the significant correlations with the MCAT verbal reasoning, the N-D 
Vocabulary, N-D Comprehension, and N-D Total scores were positively correlated with the MCAT 
physical sciences and MCAT biological sciences scores (all p = <.01). The multiple regression analyses 
supported the viability of N-D Vocabulary, N-D Comprehension, and N-D Reading Rate as predictors of 
MCAT verbal reasoning scores for medical students. 

The N-D Vocabulary and N-D Total scores were positively correlated with the USMLE Step 1 
score (both p <.01). The N-D Comprehension score was not significantly related to the USMLE Step 1 
score. The MCAT scores (verbal reasoning, physical sciences, and biological sciences) that are taken 
prior to admission to medical school show a significant relationship to the USMLE Step 1 score that is 
taken at the end of the second year of medical school. Given the observed positive relationships 
between components of the Nelson-Denny Reading Test with components of the MCAT and between 
the MCAT with the USMLE Step 1 exam, it is reasonable to suggest that medical schools should 
consider using an index of reading to better predict those applicants who will achieve success in medical 
school. As reported earlier, many factors contribute to academic success for medical students (Harward 
et al., 1998; Horshl & Kozeny, 1997). Reading ability (vocabulary, comprehension, and rate) is another 
factor to consider. 
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The 572 medical students were divided into Quartiles 1, 2, 3, and 4 based on their N-D Total 
Score. The analysis of variance with MCAT verbal reasoning as the dependent variable yielded a huge 
F-value. Once again, the powerful relationship between N-D Total and MCAT verbal reasoning was 
demonstrated. 

Medical schools recruit highly capable individuals. By being selective, they are able to 
maximize success and minimize loss. But even a small proportion of unsuccessful students, common to 
every program, has a negative impact on medical education. The N-D Total and MCAT verbal 
reasoning scores undoubtedly tap into Spearman’s “g” factor of intelligence. An extremely modest 
investment of financial and time resources in the Nelson-Denny Reading Test can yield disproportionate 
dividends for medical programs. 
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